Team Foundation Server 2010 Proxy: Informal Performance Evaluation

After launching our Team Foundation Server 2010 Proxy beta service, I was very curious to see what type of real world results would be returned. My expectations were pretty high. During the deployment process, I was just using a generic ASP.NET 4.0 Web Application Project and the improved download speed was very impressive but how about a real project?

Evaluation Method

To get an idea of what type of actual performance to expect, there isn’t any isolation involved and I’ll be using actual production servers. I’ll be using our shared TFS server hosted at our European data center and using the TFS Proxy server at our North American data center.

Instead of trying to determine connection-related efficiency from different types of Internet connections and speeds, I elected to use the office connection during normal business hours where other staff members would be utilizing the connection. I did try a few of the tests on a residential DSL connection. I will not be including the results but I’ll mention my findings in the conclusion.

There wasn’t an easy way to get start and completion times through Visual Studio – I used the tf command-line utility in a batch file. The results do not take into consideration any overhead that an IDE might require. The tests were run back-to-back so a test without proxy was immediately followed up by a request with TFS Proxy involved.

I’m also looking at two of the most frequently used commands that benefit from TFS Proxy: get and diff. Each test was run five times, a new Workspace was created for each trial and only the local client cache was cleared.

The result times were rounded up to the nearest second, as the main purpose is to try and get an idea of real world wait times.

Test Setup

Team Project Collection:

• Team Projects: 2
• Size: 44.1 MB
• Folder count: 504
• File count: 3,983

Team Foundation Server 2010 server:

• Server: ETFS01 (shared production server)
• Location: London
• Operating system: Windows Server 2008 R2 Standard (x64)
• CPU: Intel Xeon E5520
• Memory: 4GB
• Hard drive: Microsoft Virtual HD
• Connection type: Tier 1 data provider
• Average latency: 150-200 ms

Team Foundation Server 2010 Proxy server:

• Server: USW-PROXY01 (shared production server)
• Location: Irvine, CA
• Operating system: Windows Server 2008 R2 Standard (x64)
• CPU: Intel Xeon E5620
• Memory: 4GB
• Hard drive: Microsoft Virtual HD
• Connection: Tier 1 data provider
• Average latency: 7-9 ms

Team Foundation Server 2010 client (my trusty laptop!):

• Location: Pasadena, CA
• Operating system: Windows 7 Professional (x64)
• CPU: Intel Core i5-430M
• Memory: 8GB
• Hard drive: Seagate Momentus XT ST750LX003
• Connection: Business-class 10mb

Commands:

• Get (with and without Proxy):
  • Full recursive
  • Single binary file (approximately 1MB)
  • Single text file (approximately 66KB)
• Diff (with and without Proxy):
  • Single file (approximately 494KB)

Get (Full Recursive)

While it’s not something that you might end up doing frequently, I thought it would be fun just to see how much of a difference TFS Proxy makes on a full recursive get.

The recursive get of the Team Project results in downloading files from 30 seconds to a full minute faster! Again, it’s not something that would be done frequently but it’s a good way to show how much of an improvement Proxy provides.

Get (Binary)

There might be some binary files that you might end up pulling from a server. Much like the previous test, this isn’t something that you might end up doing occasionally.

TFS Proxy wound up being as much as 2.5 times faster than no proxy. I wasn’t able to try a larger file, but the benefits of using Proxy are evident.

Get (Text)

For a more practical test, I tried performing a get on a single file.

On a couple of the trials there wasn’t much of a difference since text data downloads relatively quickly. If you’re pulling a large group of files, you’d definitely see improved performance.

Diff

File comparing is a command that’s also issued frequently against a server.

Much like the get performed on an ASCII file, text-based data seems to only have a minimal benefit but I suspect that under low bandwidth high latency conditions, the difference in speed would be more noticeable.

Conclusion

After spending a lot more time working with TFS Proxy, the difference between an operation completing may seem minimal but over the course of time, the amount of time saved for a download is incredible. I’m very impressed by the performance.

The results I came across led me to believe that the ideal time to use Proxy would be in low bandwidth high latency environment so I tried performing a few of the tests from home over a residential DSL connection. After purposely reducing the transfer speed by saturating the connection with some downloads and video streams, a get against a text file was reduced from 8-9 seconds down to a very good 4-5 seconds.

I thought I’d include the statistics on the Proxy server here as well and here are the details:

• Cache size: 38MB
• Files in cache: 4,060
• Number of requests: 28,167
• Number of cache hits: 24,107
• Number of cache misses: 4,060
• Cache hits percentage: 85.59%
• Cache miss percentage: 14.41%

Proxy was used almost 86% of the entire time.

If you’ve established an account and your TFS server isn’t in your region, I’d definitely recommend enabling the Proxy service if there’s a Proxy server near you.